Analysis of the Internal Neural Networks for Ma

The internal representation of the training patterns of multi-layer perceptrons was examined and we demonstrated that the connection weights between layers are effectively transforming the representation format of the information from one layer to another one in a meaningful way. The internal code, which can be in analog or binary form, is found to be dependent on a number of factors, including the choice of an appropriate representation of the training patterns, the similarities between the patterns as well as the network structure; i.e. the number of hidden layers and the number of hidden units in each layer. In supervised neural networks, such as multi-layer perceptrons [Rumelhart, Hinton & Williams 19861, information is acquired by presenting some training examples to the network in the training process. These examples are pairs of input and output patterns. A set of connection weights is then found iteratively using the generalised delta rule and it is reserved for the classification process in the recalling phase. At present, there is no explicit guide-lines for both the choice of the size of the network and the representation format of the training examples. Trial and error has been used to decide the number of hidden layers and the number hidden units in each layer. Previous studies have shown that the number of hidden units in a multi-layer perceptron affects the performance of the network. For examples, the convergence speed and the recognition rate vary with the number of hidden units [Burr 19881. In this paper, the approach of regarding the hidden layers as the transformation process in the hyper-space were used. We illustrate that a back propagation network with internal layers solves some classification problem intelligently by using this transformation idea. In addition, we show that the internal representation of information can be affected by some characteristics of the training patterns and the architecture of the network.